start: Experiments Trail #2574

iesahin · 2021-06-21T10:31:41Z

This PR is a WIP that updates https://dvc.org/doc/start/experiments

Keeps run / show / apply and removes the rest of the material
Uses get-started-experiments in command examples
Some commands may need an update after Add image directory version of Fashion-MNIST dataset dataset-registry#18
Closes start: A new GS:Experiments document with get-started-experiments #2479
Replaces start: use get-started-experiments for GS:Experiments tutorial #2497
Related start: independent trails #2496

Waiting For

TODO

Select one of the draft documents
Add to sidebar

iesahin · 2021-06-21T10:36:27Z

I used the following map to write the document:

shcheklein · 2021-06-23T02:22:30Z

Suggestion - let's not modify the existing file. Let's start a fresh doc/section (name it "Experiments Trail"), so that there is no confusion - it will be easier to review.

content/docs/start/experiments.md

jorgeorpinel · 2021-06-28T23:54:07Z

Let's start a fresh doc/section (name it "Experiments Trail"), so that there is no confusion - it will be easier to review

➕ 1️⃣ on this. Is that the plan? It can be a useful trick even if you end up removing the existing file and reusing it's name for the new one just before merging.

Another option that may work is if you remove the file in one commit and then add it again with totally different content in another commit: hopefully GH won't mix in the changes and will just show all red on top and all green in the bottom.

iesahin · 2021-07-06T16:55:49Z

We now have two drafts for the new GS:Experiments.

~~First of them~~ uses dvc get and dvc stage add to create a pipeline, then proceeds to experiments.

~~The second one~~ starts from an already prepared pipeline and uses dvc pull to get the data dependency.

PTAL and let me know which one is the route to proceed. I've also added a paragraph about dvc exp branch and a section about the experiment naming. Thank you.

Edit: Links are renewed in every push. I put them to the sidebar.

@shcheklein @dberenbaum @jorgeorpinel

shcheklein · 2021-07-06T23:09:04Z

~~@iesahin could we add both to the side bar for now? to simplify the review? :)~~

I see the links now 🤦

jorgeorpinel · 2021-07-07T04:59:38Z

First of them uses dvc get and dvc stage add to create a pipeline
The second one starts from an already prepared pipeline

From my PoV this is important but secondary, so I'll look at the main content for now ⌛ (I understand that's the same in both alternatives).

content/docs/start/experiments-trail/experiments-2.md

jorgeorpinel

Overall it seems like all that planning and maps are paying off @iesahin! I'm liking the simplicity and structure. I think it still needs some work (see review 👇) but great first version 👍

There's also a bunch of minor details but I'll make specific suggestions for those in a later round.

content/docs/start/experiments-trail/experiments-1.md

Co-authored-by: Julie <[email protected]>

iesahin · 2021-10-14T08:02:07Z

Rebased to master. Waiting for one last review from @dberenbaum :)

dberenbaum

Looks good! Pretty much one substantive comment and lots of little grammatical suggestions since it's a final review.

content/docs/start/experiments.md

….org into iesahin/new-gs-experiments

dberenbaum

Found a few more minor suggestions reading through it again. Once those are resolved, I can approve.

dberenbaum · 2021-10-15T18:18:22Z

content/docs/start/experiments.md

+In machine learning projects, the number of <abbr>experiments</abbr> grows
+rapidly. DVC can track these experiments, list and compare their most relevant
+parameters and metrics, navigate among them, and commit only the ones that we
+need to Git.

 https://youtu.be/FHQq_zZz5ms


Are we keeping this even though it doesn't align with the text?

I prefer text to video in any case, but for those who prefer video to text always, it may be better to keep it. I have no strong opinions on this though, we can remove it. WDYT @shcheklein?

I created #2935 to renew the video.

If we have doubts we could put a note for now nearby the video saying that it's not exactly the same and a bit outdated and will be updated soon, wdyt?

content/docs/start/experiments.md

iesahin · 2021-10-18T14:58:01Z

Thank you so much @dberenbaum. Revised per your review.

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--oepxvh June 21, 2021 10:31 Inactive

restyled-io bot mentioned this pull request Jun 21, 2021

Restyle start: new experiments introduction #2575

Closed

iesahin marked this pull request as draft June 21, 2021 10:32

iesahin changed the title ~~start: new experiments introduction~~ [WIP] start: new experiments introduction Jun 21, 2021

iesahin changed the title ~~[WIP] start: new experiments introduction~~ [WIP] start: new experiments Jun 22, 2021

jorgeorpinel reviewed Jun 23, 2021

View reviewed changes

content/docs/start/experiments.md Outdated Show resolved Hide resolved

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--nn6jh0 July 2, 2021 13:04 Inactive

iesahin changed the title ~~[WIP] start: new experiments~~ [WIP] start: experiments trail drafts 1 & 2 Jul 2, 2021

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--ktwsba July 3, 2021 10:46 Inactive

restyled-io bot mentioned this pull request Jul 5, 2021

Restyle [WIP] start: experiments trail drafts 1 & 2 #2608

Merged

shcheklein had a problem deploying to dvc-org-iesahin-new-gs--pgcplz July 5, 2021 19:36 Failure

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--pgcplz July 5, 2021 19:45 Inactive

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--lq3od2 July 6, 2021 09:55 Inactive

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--lq3od2 July 6, 2021 10:21 Inactive

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--lq3od2 July 6, 2021 15:38 Inactive

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--lq3od2 July 6, 2021 15:46 Inactive

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--lq3od2 July 6, 2021 16:24 Inactive

iesahin marked this pull request as ready for review July 6, 2021 16:27

iesahin marked this pull request as draft July 6, 2021 16:27

jorgeorpinel reviewed Jul 7, 2021

View reviewed changes

content/docs/start/experiments-trail/experiments-2.md Outdated Show resolved Hide resolved

jorgeorpinel reviewed Jul 7, 2021

View reviewed changes

iesahin mentioned this pull request Jul 7, 2021

start: use get-started-experiments for GS:Experiments tutorial #2497

Closed

jorgeorpinel assigned iesahin Jul 7, 2021

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--lq3od2 July 8, 2021 09:50 Inactive

iesahin and others added 12 commits October 14, 2021 10:55

revised the doc per reviews

b7d37dc

Update content/docs/start/experiments-trail/experiments.md

4e81e46

Co-authored-by: Julie <[email protected]>

minor edits

02cd702

some emoji changes

62edbd8

put the video

10e6ec3

replaced older gs:experiments with the trail doc

bd4b6ad

Removed emoji and moved persisting to details section

8dfe709

added a sentence for data management

56ccd4c

added a brief dvc exp show after the first run.

2a8b42e

added emoji to detail headers

769c2e6

added an initial text

c7ddcf8

some suggestions of Ivan applied.

915c61a

iesahin force-pushed the iesahin/new-gs-experiments branch from 72cca7c to 915c61a Compare October 14, 2021 08:00

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--gh4o2v October 14, 2021 08:00 Inactive

iesahin mentioned this pull request Oct 14, 2021

start: integrate (updated) params/metrics/plots to Experiments Trail #2925

Closed

3 tasks

dberenbaum suggested changes Oct 14, 2021

View reviewed changes

jorgeorpinel assigned karajan1001 and unassigned karajan1001 Oct 15, 2021

iesahin added 3 commits October 15, 2021 20:02

modifications after Dave's review

ea94da6

Merge branch 'iesahin/new-gs-experiments' of github.com:iterative/dvc…

2b9ebd9

….org into iesahin/new-gs-experiments

merged

ec3d132

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--gh4o2v October 15, 2021 17:06 Inactive

dberenbaum suggested changes Oct 15, 2021

View reviewed changes

revised after Dave's review.

d3cae7a

shcheklein temporarily deployed to dvc-org-iesahin-new-gs--gh4o2v October 18, 2021 14:56 Inactive

dberenbaum approved these changes Oct 18, 2021

View reviewed changes

shcheklein merged commit 55abb93 into master Oct 18, 2021

shcheklein deleted the iesahin/new-gs-experiments branch October 18, 2021 18:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

start: Experiments Trail #2574

start: Experiments Trail #2574

iesahin commented Jun 21, 2021 •

edited

Loading

iesahin commented Jun 21, 2021

shcheklein commented Jun 23, 2021

jorgeorpinel commented Jun 28, 2021

iesahin commented Jul 6, 2021 •

edited

Loading

shcheklein commented Jul 6, 2021 •

edited

Loading

jorgeorpinel commented Jul 7, 2021 •

edited

Loading

jorgeorpinel left a comment

iesahin commented Oct 14, 2021

dberenbaum left a comment

dberenbaum left a comment

dberenbaum Oct 15, 2021

iesahin Oct 18, 2021

shcheklein Oct 18, 2021

iesahin commented Oct 18, 2021

start: Experiments Trail #2574

start: Experiments Trail #2574

Conversation

iesahin commented Jun 21, 2021 • edited Loading

Waiting For

TODO

iesahin commented Jun 21, 2021

shcheklein commented Jun 23, 2021

jorgeorpinel commented Jun 28, 2021

iesahin commented Jul 6, 2021 • edited Loading

shcheklein commented Jul 6, 2021 • edited Loading

jorgeorpinel commented Jul 7, 2021 • edited Loading

jorgeorpinel left a comment

Choose a reason for hiding this comment

iesahin commented Oct 14, 2021

dberenbaum left a comment

Choose a reason for hiding this comment

dberenbaum left a comment

Choose a reason for hiding this comment

dberenbaum Oct 15, 2021

Choose a reason for hiding this comment

iesahin Oct 18, 2021

Choose a reason for hiding this comment

shcheklein Oct 18, 2021

Choose a reason for hiding this comment

iesahin commented Oct 18, 2021

iesahin commented Jun 21, 2021 •

edited

Loading

iesahin commented Jul 6, 2021 •

edited

Loading

shcheklein commented Jul 6, 2021 •

edited

Loading

jorgeorpinel commented Jul 7, 2021 •

edited

Loading